Update ll.yml #20
Merged
Update ll.yml #20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR |
|
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR |
|
This issue is available for anyone to work on. Make sure to reference this issue in your pull request. ✨ Thank you for your contribution! ✨ |
|
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR |
CalvinKirs
pushed a commit
that referenced
this pull request
Jun 11, 2024
…l_backend() (apache#35634) ## Proposed changes ```cpp ==1825462==ERROR: AddressSanitizer: heap-use-after-free on address 0x5030012ea880 at pc 0x555cc645e1aa bp 0x7fe4a9407e20 sp 0x7fe4a94075e0 READ of size 22 at 0x5030012ea880 thread T2250 (REPORT_DISK_STA) #0 0x555cc645e1a9 in __asan_memcpy (/mnt/disk1/xiaolei/incubator-doris/output/be/lib/doris_be+0x22cb01a9) (BuildId: 76fba423c52f4da5) #1 0x555cc64b6990 in std::char_traits<char>::copy(char*, char const*, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/char_traits.h:445:33 #2 0x555cc64b68a1 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_S_copy(char*, char const*, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:420:4 #3 0x555cc64c526e in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:291:6 #4 0x555cc64c5070 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:1596:8 #5 0x555cc64b1d9c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:802:15 #6 0x555ccb05a722 in doris::TBackend::TBackend(doris::TBackend const&) /mnt/disk1/xiaolei/incubator-doris/gensrc/build/gen_cpp/Types_types.cpp:5755:8 #7 0x555cc9c9234e in doris::BackendOptions::get_local_backend() /mnt/disk1/xiaolei/incubator-doris/be/src/service/backend_options.cpp:79:12 #8 0x555cc6586e5a in doris::report_disk_callback(doris::StorageEngine&, doris::TMasterInfo const&) /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:1021:27 #9 0x555cc64f2a0a in doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/agent_server.cpp:190:133 #10 0x555cc64f2994 in void std::__invoke_impl<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>(std::__invoke_other, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #11 0x555cc64f2944 in std::enable_if<is_invocable_r_v<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>, void>::type std::__invoke_r<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>(doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #12 0x555cc64f280c in std::_Function_handler<void (), doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #13 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #14 0x555cc65b01f9 in doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:689:13 #15 0x555cc65afcd4 in void std::__invoke_impl<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(std::__invoke_other, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #16 0x555cc65afc74 in std::enable_if<is_invocable_r_v<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>, void>::type std::__invoke_r<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #17 0x555cc65afa6c in std::_Function_handler<void (), doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #18 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #19 0x555cca138dab in doris::Thread::supervise_thread(void*) /mnt/disk1/xiaolei/incubator-doris/be/src/util/thread.cpp:498:5 #20 0x555cc645de0a in asan_thread_start(void*) crtstuff.c #21 0x7fef061601c9 in start_thread (/lib64/libpthread.so.0+0x81c9) (BuildId: 823fccea3475e5870a4167dfe47df20e53222db0) #22 0x7fef06b4fe72 in clone (/lib64/libc.so.6+0x39e72) (BuildId: ec3d7025354f1f1985831ff08ef0eb3b50aefbce) 0x5030012ea880 is located 0 bytes inside of 31-byte region [0x5030012ea880,0x5030012ea89f) freed by thread T2249 (REPORT_TASK-182) here: #0 0x555cc649d43d in operator delete(void*) (/mnt/disk1/xiaolei/incubator-doris/output/be/lib/doris_be+0x22cef43d) (BuildId: 76fba423c52f4da5) #1 0x555cc64b6bbc in std::__new_allocator<char>::deallocate(char*, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/new_allocator.h:168:2 #2 0x555cc64b6b5d in std::allocator<char>::deallocate(char*, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/allocator.h:210:25 #3 0x555cc64b6b5d in std::allocator_traits<std::allocator<char>>::deallocate(std::allocator<char>&, char*, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/alloc_traits.h:516:13 #4 0x555cc64b6b5d in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_destroy(unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:289:9 #5 0x555cc64b6a56 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_dispose() /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:283:4 #6 0x555cc64c51ee in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:285:8 #7 0x555cc64c5070 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:1596:8 #8 0x555cc64b1d9c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:802:15 #9 0x555ccb058510 in doris::TBackend::__set_host(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/incubator-doris/gensrc/build/gen_cpp/Types_types.cpp:5584:14 #10 0x555cc9c92239 in doris::BackendOptions::get_local_backend() /mnt/disk1/xiaolei/incubator-doris/be/src/service/backend_options.cpp:75:14 #11 0x555cc6585b9d in doris::report_task_callback(doris::TMasterInfo const&) /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:1004:27 #12 0x555cc64f24b9 in doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/agent_server.cpp:187:112 #13 0x555cc64f2474 in void std::__invoke_impl<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20&>(std::__invoke_other, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #14 0x555cc64f2424 in std::enable_if<is_invocable_r_v<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20&>, void>::type std::__invoke_r<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20&>(doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #15 0x555cc64f22ec in std::_Function_handler<void (), doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #16 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #17 0x555cc65b01f9 in doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:689:13 #18 0x555cc65afcd4 in void std::__invoke_impl<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(std::__invoke_other, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #19 0x555cc65afc74 in std::enable_if<is_invocable_r_v<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>, void>::type std::__invoke_r<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #20 0x555cc65afa6c in std::_Function_handler<void (), doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #21 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #22 0x555cca138dab in doris::Thread::supervise_thread(void*) /mnt/disk1/xiaolei/incubator-doris/be/src/util/thread.cpp:498:5 #23 0x555cc645de0a in asan_thread_start(void*) crtstuff.c previously allocated by thread T2250 (REPORT_DISK_STA) here: #0 0x555cc649cbdd in operator new(unsigned long) (/mnt/disk1/xiaolei/incubator-doris/output/be/lib/doris_be+0x22ceebdd) (BuildId: 76fba423c52f4da5) #1 0x555cc64b684e in std::__new_allocator<char>::allocate(unsigned long, void const*) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/new_allocator.h:147:27 #2 0x555cc64b67b0 in std::allocator<char>::allocate(unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/allocator.h:198:32 #3 0x555cc64b67b0 in std::allocator_traits<std::allocator<char>>::allocate(std::allocator<char>&, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/alloc_traits.h:482:20 #4 0x555cc64b67b0 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_S_allocate(std::allocator<char>&, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:126:16 #5 0x555cc64b63b1 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_create(unsigned long&, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:155:14 #6 0x555cc64c51de in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:284:24 #7 0x555cc64c5070 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:1596:8 #8 0x555cc64b1d9c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:802:15 #9 0x555ccb058510 in doris::TBackend::__set_host(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/incubator-doris/gensrc/build/gen_cpp/Types_types.cpp:5584:14 #10 0x555cc9c92239 in doris::BackendOptions::get_local_backend() /mnt/disk1/xiaolei/incubator-doris/be/src/service/backend_options.cpp:75:14 #11 0x555cc6586e5a in doris::report_disk_callback(doris::StorageEngine&, doris::TMasterInfo const&) /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:1021:27 #12 0x555cc64f2a0a in doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/agent_server.cpp:190:133 #13 0x555cc64f2994 in void std::__invoke_impl<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>(std::__invoke_other, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #14 0x555cc64f2944 in std::enable_if<is_invocable_r_v<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>, void>::type std::__invoke_r<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>(doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #15 0x555cc64f280c in std::_Function_handler<void (), doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #16 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #17 0x555cc65b01f9 in doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:689:13 #18 0x555cc65afcd4 in void std::__invoke_impl<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(std::__invoke_other, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #19 0x555cc65afc74 in std::enable_if<is_invocable_r_v<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>, void>::type std::__invoke_r<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #20 0x555cc65afa6c in std::_Function_handler<void (), doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #21 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #22 0x555cca138dab in doris::Thread::supervise_thread(void*) /mnt/disk1/xiaolei/incubator-doris/be/src/util/thread.cpp:498:5 #23 0x555cc645de0a in asan_thread_start(void*) crtstuff.c ```
CalvinKirs
pushed a commit
that referenced
this pull request
Jun 19, 2024
…t is not nullable (apache#36263) ## Proposed changes For a null safe equal join, FE may generate a plan that both sides of the conjuct are not nullable, which is a valid plan, BE will coredump in this situation. Issue Number: close #xxx Coredump stacktrace: ``` #0 0x00007fff042caacf in raise () from /lib64/libc.so.6 #1 0x00007fff0429dea5 in abort () from /lib64/libc.so.6 #2 0x00005555a7ec7cad in ?? () #3 0x00005555a7eba34a in google::LogMessage::Fail() () #4 0x00005555a7ebd436 in google::LogMessage::SendToLog() () #5 0x00005555a7eb9e80 in google::LogMessage::Flush() () #6 0x00005555a7ebdc79 in google::LogMessageFatal::~LogMessageFatal() () #7 0x000055557824c9ac in doris::vectorized::IColumn::get_raw_data (this=0x60400387df90) at /mnt/disk2/tengjianping/doris-39/be/src/vec/columns/column.h:590 #8 0x00005555a4a85f32 in doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>::pack_fixeds<unsigned long> (this=0x6110086dcb50, row_numbers=21, key_columns=std::vector of length 1, capacity 1 = {...}, nullmap_columns=std::vector of length 0, capacity 0, result=std::vector of length 21, capacity 21 = {...}) at /mnt/disk2/tengjianping/doris-39/be/src/vec/common/hash_table/hash_map_context.h:405 #9 0x00005555a4a83a77 in doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>::init_serialized_keys (this=0x6110086dcb50, key_columns=std::vector of length 1, capacity 1 = {...}, num_rows=21, null_map=0x0, is_join=true, is_build=true, bucket_size=32) at /mnt/disk2/tengjianping/doris-39/be/src/vec/common/hash_table/hash_map_context.h:470 #10 0x00005555a48d9228 in doris::pipeline::ProcessHashTableBuild<doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false> >::run<1, false, false, false> (this=0x75d6195d92a0, hash_table_ctx=..., null_map=0x0, has_null_key=0x6120057db3c0) at /mnt/disk2/tengjianping/doris-39/be/src/pipeline/exec/hashjoin_build_sink.h:208 #11 0x00005555a456dd23 in doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeS tate*, doris::vectorized::Block&)::$_1::operator()<doris::vectorized::MethodKeysFixed<doris::JoinHashTable< unsigned long, HashCRC32<unsigned long> >, false>&, std::integral_constant<do--Type <RET> for more, q to qu --Type <RET> for more, q to quit, c to continue without paging--c ris::TJoinOp::type, (doris::TJoinOp::type)1>&, std::integral_constant<bool, false>, std::integral_constant<bool, false>, std::integral_constant<bool, false> >(doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>&, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)1>&, std::integral_constant<bool, false>, std::integral_constant<bool, false>, std::integral_constant<bool, false>) const (this=0x75d61989aa10, arg=..., join_op=..., has_null_value=..., short_circuit_for_null_in_build_side=..., with_other_conjuncts=...) at /mnt/disk2/tengjianping/doris-39/be/src/pipeline/exec/hashjoin_build_sink.cpp:296 #12 0x00005555a456d93c in std::__invoke_impl<doris::Status, doris::vectorized::Overload<doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_0, doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_1>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>&, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)1>&, std::integral_constant<bool, false>, std::integral_constant<bool, false>, std::integral_constant<bool, false> >(std::__invoke_other, doris::vectorized::Overload<doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_0, doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_1>&&, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>&, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)1>&, std::integral_constant<bool, false>&&, std::integral_constant<bool, false>&&, std::integral_constant<bool, false>&&) (__f=..., __args=..., __args=..., __args=..., __args=..., __args=...) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib64/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61 #13 0x00005555a456d88c in std::__invoke<doris::vectorized::Overload<doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_0, doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_1>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>&, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)1>&, std::integral_constant<bool, false>, std::integral_constant<bool, false>, std::integral_constant<bool, false> >(doris::vectorized::Overload<doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_0, doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_1>&&, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>&, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)1>&, std::integral_constant<bool, false>&&, std::integral_constant<bool, false>&&, std::integral_constant<bool, false>&&) (__fn=..., __args=..., __args=..., __args=..., __args=..., __args=...) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib64/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96 #14 0x00005555a43eef75 in _ZNSt8__detail9__variant17__gen_vtable_implINS0_12_Multi_arrayIPFNS0_21__deduce_visit_resultIN5doris6StatusEEEONS4_10vectorized8OverloadIJZNS4_8pipeline27HashJoinBuildSinkLocalState19process_build_blockEPNS4_12RuntimeStateERNS7_5BlockEE3$_0ZNSA_19process_build_blockESC_SE_E3$_1EEERSt7variantIJSt9monostateNS7_16MethodSerializedINS4_13JoinHashTableINS4_9StringRefE11DefaultHashISN_vEEEEENS7_15MethodOneNumberIhNSM_Ih9HashCRC32IhEEEEENSS_ItNSM_ItST_ItEEEEENSS_IjNSM_IjST_IjEEEEENSS_ImNSM_ImST_ImEEEEENSS_IN4wide7integerILm128EjEENSM_IS18_ST_IS18_EEEEENSS_INS17_ILm256EjEENSM_IS1C_ST_IS1C_EEEEENS7_15MethodKeysFixedIS14_Lb1EEENS1G_IS14_Lb0EEENS1G_IS1A_Lb1EEENS1G_IS1A_Lb0EEENS1G_IS1E_Lb1EEENS1G_IS1E_Lb0EEENS1G_INSM_INS7_7UInt136EST_IS1N_EEELb1EEENS1G_IS1P_Lb0EEEEERSJ_IJSt17integral_constantINS4_7TJoinOp4typeELS1W_0EES1U_IS1W_LS1W_2EES1U_IS1W_LS1W_8EES1U_IS1W_LS1W_1EES1U_IS1W_LS1W_4EES1U_IS1W_LS1W_3EES1U_IS1W_LS1W_5EES1U_IS1W_LS1W_7EES1U_IS1W_LS1W_9EES1U_IS1W_LS1W_10EES1U_IS1W_LS1W_11EEEEOSJ_IJS1U_IbLb0EES1U_IbLb1EEEES2D_S2D_EJEEESt16integer_sequenceImJLm9ELm3ELm0ELm0ELm0EEEE14__visit_invokeESI_S1T_S29_S2D_S2D_S2D_ (__visitor=..., __vars=..., __vars=..., __vars=..., __vars=..., __vars=...) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib64/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/variant:1013 #15 0x00005555a43cf068 in std::__do_visit<std::__detail::__variant::__deduce_visit_result<doris::Status>, doris::vectorized::Overload<doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_0, doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_1>, std::variant<std::monostate, doris::vectorized::MethodSerialized<doris::JoinHashTable<doris::StringRef, DefaultHash<doris::StringRef, void> > >, doris::vectorized::MethodOneNumber<unsigned char, doris::JoinHashTable<unsigned char, HashCRC32<unsigned char> > >, doris::vectorized::MethodOneNumber<unsigned short, doris::JoinHashTable<unsigned short, HashCRC32<unsigned short> > >, doris::vectorized::MethodOneNumber<unsigned int, doris::JoinHashTable<unsigned int, HashCRC32<unsigned int> > >, doris::vectorized::MethodOneNumber<unsigned long, doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> > >, doris::vectorized::MethodOneNumber<wide::integer<128ul, unsigned int>, doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > > >, doris::vectorized::MethodOneNumber<wide::integer<256ul, unsigned int>, doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > > >, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<doris::vectorized::UInt136, HashCRC32<doris::vectorized::UInt136> >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<doris::vectorized::UInt136, HashCRC32<doris::vectorized::UInt136> >, false> >&, std::variant<std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)0>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)2>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)8>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)1>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)4>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)3>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)5>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)7>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)9>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)10>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)11> >&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> >, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> >, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> > >(doris::vectorized::Overload<doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_0, doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_1>&&, std::variant<std::monostate, doris::vectorized::MethodSerialized<doris::JoinHashTable<doris::StringRef, DefaultHash<doris::StringRef, void> > >, doris::vectorized::MethodOneNumber<unsigned char, doris::JoinHashTable<unsigned char, HashCRC32<unsigned char> > >, doris::vectorized::MethodOneNumber<unsigned short, doris::JoinHashTable<unsigned short, HashCRC32<unsigned short> > >, doris::vectorized::MethodOneNumber<unsigned int, doris::JoinHashTable<unsigned int, HashCRC32<unsigned int> > >, doris::vectorized::MethodOneNumber<unsigned long, doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> > >, doris::vectorized::MethodOneNumber<wide::integer<128ul, unsigned int>, doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > > >, doris::vectorized::MethodOneNumber<wide::integer<256ul, unsigned int>, doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > > >, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<doris::vectorized::UInt136, HashCRC32<doris::vectorized::UInt136> >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<doris::vectorized::UInt136, HashCRC32<doris::vectorized::UInt136> >, false> >&, std::variant<std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)0>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)2>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)8>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)1>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)4>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)3>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)5>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)7>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)9>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)10>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)11> >&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> >&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> >&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> >&&) (__visitor=..., __variants=..., __variants=..., __variants=..., __variants=..., __variants=...) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib64/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/variant:1714 #16 0x00005555a43c8dea in std::visit<doris::vectorized::Overload<doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_0, doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_1>, std::variant<std::monostate, doris::vectorized::MethodSerialized<doris::JoinHashTable<doris::StringRef, DefaultHash<doris::StringRef, void> > >, doris::vectorized::MethodOneNumber<unsigned char, doris::JoinHashTable<unsigned char, HashCRC32<unsigned char> > >, doris::vectorized::MethodOneNumber<unsigned short, doris::JoinHashTable<unsigned short, HashCRC32<unsigned short> > >, doris::vectorized::MethodOneNumber<unsigned int, doris::JoinHashTable<unsigned int, HashCRC32<unsigned int> > >, doris::vectorized::MethodOneNumber<unsigned long, doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> > >, doris::vectorized::MethodOneNumber<wide::integer<128ul, unsigned int>, doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > > >, doris::vectorized::MethodOneNumber<wide::integer<256ul, unsigned int>, doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > > >, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<doris::vectorized::UInt136, HashCRC32<doris::vectorized::UInt136> >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<doris::vectorized::UInt136, HashCRC32<doris::vectorized::UInt136> >, false> >&, std::variant<std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)0>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)2>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)8>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)1>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)4>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)3>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)5>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)7>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)9>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)10>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)11> >&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> >, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> >, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> > >(doris::vectorized::Overload<doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_0, doris::pipeline::HashJoinBuildSinkLocalState::process_build_block(doris::RuntimeState*, doris::vectorized::Block&)::$_1>&&, std::variant<std::monostate, doris::vectorized::MethodSerialized<doris::JoinHashTable<doris::StringRef, DefaultHash<doris::StringRef, void> > >, doris::vectorized::MethodOneNumber<unsigned char, doris::JoinHashTable<unsigned char, HashCRC32<unsigned char> > >, doris::vectorized::MethodOneNumber<unsigned short, doris::JoinHashTable<unsigned short, HashCRC32<unsigned short> > >, doris::vectorized::MethodOneNumber<unsigned int, doris::JoinHashTable<unsigned int, HashCRC32<unsigned int> > >, doris::vectorized::MethodOneNumber<unsigned long, doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> > >, doris::vectorized::MethodOneNumber<wide::integer<128ul, unsigned int>, doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > > >, doris::vectorized::MethodOneNumber<wide::integer<256ul, unsigned int>, doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > > >, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<unsigned long, HashCRC32<unsigned long> >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<128ul, unsigned int>, HashCRC32<wide::integer<128ul, unsigned int> > >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<wide::integer<256ul, unsigned int>, HashCRC32<wide::integer<256ul, unsigned int> > >, false>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<doris::vectorized::UInt136, HashCRC32<doris::vectorized::UInt136> >, true>, doris::vectorized::MethodKeysFixed<doris::JoinHashTable<doris::vectorized::UInt136, HashCRC32<doris::vectorized::UInt136> >, false> >&, std::variant<std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)0>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)2>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)8>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)1>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)4>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)3>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)5>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)7>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)9>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)10>, std::integral_constant<doris::TJoinOp::type, (doris::TJoinOp::type)11> >&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> >&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> >&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true> >&&) (__visitor=..., __variants=..., __variants=..., __variants=..., __variants=..., __variants=...) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib64/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/variant:1769 #17 0x00005555a43c848d in doris::pipeline::HashJoinBuildSinkLocalState::process_build_block (this=0x616000669080, state=0x61e003eca080, block=...) at /mnt/disk2/tengjianping/doris-39/be/src/pipeline/exec/hashjoin_build_sink.cpp:278 #18 0x00005555a43cd0a2 in doris::pipeline::HashJoinBuildSinkOperatorX::sink (this=0x613001702bc0, state=0x61e003eca080, in_block=0x60d0000f7d40, eos=true) at /mnt/disk2/tengjianping/doris-39/be/src/pipeline/exec/hashjoin_build_sink.cpp:557 #19 0x00005555a7bdd4e1 in doris::pipeline::PipelineTask::execute(bool*)::$_1::operator()() const (this=0x75d619925bd0) at /mnt/disk2/tengjianping/doris-39/be/src/pipeline/pipeline_task.cpp:356 #20 0x00005555a7bda84c in doris::pipeline::PipelineTask::execute (this=0x61700073d800, eos=0x75d6198f4230) at /mnt/disk2/tengjianping/doris-39/be/src/pipeline/pipeline_task.cpp:359 #21 0x00005555a7c34c6a in doris::pipeline::TaskScheduler::_do_work (this=0x60b00226b9e0, index=10) at /mnt/disk2/tengjianping/doris-39/be/src/pipeline/task_scheduler.cpp:135 #22 0x00005555a7c374bb in doris::pipeline::TaskScheduler::start()::$_0::operator()() const (this=0x6060055527f8) at /mnt/disk2/tengjianping/doris-39/be/src/pipeline/task_scheduler.cpp:64 ```
CalvinKirs
pushed a commit
that referenced
this pull request
Jun 19, 2024
…6275) If we execute a delete predicate in an inverted index table and then query something from it, it will cause a core dump in shrink_char_type_column_suffix_zero. This occurs because the delete predicate column ID in _char_type_idx is incorrectly shifted to the result column inserted by the inverted index result. coredump stack like: ``` COULD NOT CREATE A LOGGINGFILE 20240604-145331!F20240604 14:53:31.991016 24178 column.h:134] Cannot get_shrinked_column() column Const(UInt8) *** Check failure stack trace: *** F20240604 14:53:31.991436 24158 column.h:134] Cannot get_shrinked_column() column Const(UInt8) *** Check failure stack trace: *** F20240604 14:53:31.991436 24158 column.h:134] Cannot get_shrinked_column() column Const(UInt8)F20240604 14:53:31.991645 24160 column.h:134] Cannot get_shrinked_column() column Const(UInt8) #0 0x00007fa0bf784387 in ?? () from /mnt/disk2/tengjianping/local/ldb_toolchain/lib/libc.so.6 #1 0x0000557dc8e617a0 in google::LogMessage::Flush() () #2 0x0000557dc8e65599 in google::LogMessageFatal::~LogMessageFatal() () #3 0x0000557dbe4ec939 in doris::vectorized::IColumn::get_shrinked_column (this=0x7fa094ac16a0) at /home/zcp/repo_center/doris_release/doris/be/src/vec/columns/column.h:134 #4 0x0000557dc236d8a2 in doris::vectorized::Block::shrink_char_type_column_suffix_zero (this=0x7f9a372a4b70, char_type_idx=...) at /home/zcp/repo_center/doris_release/doris/be/src/vec/core/block.cpp:1126 #5 0x0000557dbee9ea95 in doris::segment_v2::SegmentIterator::_next_batch_internal (this=<optimized out>, block=0x7f9a372a4b70) at /home/zcp/repo_center/doris_release/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2408 #6 0x0000557dbee9cc5a in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const (this=<optimized out>) at /home/zcp/repo_center/doris_release/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2090 #7 doris::segment_v2::SegmentIterator::next_batch (this=0x1c5d, block=0x6) at /home/zcp/repo_center/doris_release/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2089 #8 0x0000557dbeebbb44 in doris::BetaRowsetReader::next_block (this=0x7f9a3a041600, block=0x7f9a372a4b70) at /home/zcp/repo_center/doris_release/doris/be/src/olap/rowset/beta_rowset_reader.cpp:342 #9 0x0000557dc8590ddd in doris::vectorized::VCollectIterator::Level0Iterator::_refresh (this=0x7f9a372a1be0) at /home/zcp/repo_center/doris_release/doris/be/src/vec/olap/vcollect_iterator.h:256 #10 doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row (this=0x7f9a372a1be0) at /home/zcp/repo_center/doris_release/doris/be/src/vec/olap/vcollect_iterator.cpp:514 #11 0x0000557dc8591055 in doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref (this=0x1c5d) at /home/zcp/repo_center/doris_release/doris/be/src/vec/olap/vcollect_iterator.cpp:493 #12 0x0000557dc85934f2 in doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref (this=0x7fa090e36c00) at /home/zcp/repo_center/doris_release/doris/be/src/vec/olap/vcollect_iterator.cpp:692 #13 0x0000557dc858ddc9 in doris::vectorized::VCollectIterator::build_heap (this=0x7f9d80976dc0, rs_readers=...) at /home/zcp/repo_center/doris_release/doris/be/src/vec/olap/vcollect_iterator.cpp:186 #14 0x0000557dc857e808 in doris::vectorized::BlockReader::_init_collect_iter (this=this@entry=0x7f9d80976800, read_params=...) at /home/zcp/repo_center/doris_release/doris/be/src/vec/olap/block_reader.cpp:156 #15 0x0000557dc857f64d in doris::vectorized::BlockReader::init (this=<optimized out>, read_params=...) at /home/zcp/repo_center/doris_release/doris/be/src/vec/olap/block_reader.cpp:228 #16 0x0000557dc370dc2c in doris::vectorized::NewOlapScanner::open (this=0x7f9a2aa68210, state=<optimized out>) --Type <RET> for more, q to quit, c to continue without paging--c at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:227 #17 0x0000557dc3718d89 in doris::vectorized::ScannerScheduler::_scanner_scan (ctx=std::shared_ptr<doris::vectorized::ScannerContext> (use count 4, weak count 1) = {...}, scan_task=std::shared_ptr<doris::vectorized::ScanTask> (use count 2, weak count 0) = {...}) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:242 #18 0x0000557dc3719b77 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const (this=0x0) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:158 #19 std::__invoke_impl<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1&) (__f=...) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61 #20 std::__invoke_r<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1&) (__fn=...) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111 #21 std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1>::_M_invoke(std::_Any_data const&) (__functor=...) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291 #22 0x0000557dbf3c7788 in doris::ThreadPool::dispatch_thread (this=0x7f9d0bbdc400) at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:543 #23 0x0000557dbf3bcb41 in std::function<void ()>::operator()() const (this=0x1471) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560 #24 doris::Thread::supervise_thread (arg=0x7f9d0bbdf4c0) at /home/zcp/repo_center/doris_release/doris/be/src/util/thread.cpp:498 #25 0x00007fa0bee1dea5 in sem_open () from /mnt/disk2/tengjianping/local/ldb_toolchain/lib/libpthread.so.0 ```
CalvinKirs
pushed a commit
that referenced
this pull request
Jul 9, 2024
…l_backend() (apache#35634) ## Proposed changes ```cpp ==1825462==ERROR: AddressSanitizer: heap-use-after-free on address 0x5030012ea880 at pc 0x555cc645e1aa bp 0x7fe4a9407e20 sp 0x7fe4a94075e0 READ of size 22 at 0x5030012ea880 thread T2250 (REPORT_DISK_STA) #0 0x555cc645e1a9 in __asan_memcpy (/mnt/disk1/xiaolei/incubator-doris/output/be/lib/doris_be+0x22cb01a9) (BuildId: 76fba423c52f4da5) #1 0x555cc64b6990 in std::char_traits<char>::copy(char*, char const*, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/char_traits.h:445:33 #2 0x555cc64b68a1 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_S_copy(char*, char const*, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:420:4 #3 0x555cc64c526e in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:291:6 #4 0x555cc64c5070 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:1596:8 #5 0x555cc64b1d9c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:802:15 #6 0x555ccb05a722 in doris::TBackend::TBackend(doris::TBackend const&) /mnt/disk1/xiaolei/incubator-doris/gensrc/build/gen_cpp/Types_types.cpp:5755:8 #7 0x555cc9c9234e in doris::BackendOptions::get_local_backend() /mnt/disk1/xiaolei/incubator-doris/be/src/service/backend_options.cpp:79:12 #8 0x555cc6586e5a in doris::report_disk_callback(doris::StorageEngine&, doris::TMasterInfo const&) /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:1021:27 #9 0x555cc64f2a0a in doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/agent_server.cpp:190:133 #10 0x555cc64f2994 in void std::__invoke_impl<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>(std::__invoke_other, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #11 0x555cc64f2944 in std::enable_if<is_invocable_r_v<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>, void>::type std::__invoke_r<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>(doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #12 0x555cc64f280c in std::_Function_handler<void (), doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #13 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #14 0x555cc65b01f9 in doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:689:13 #15 0x555cc65afcd4 in void std::__invoke_impl<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(std::__invoke_other, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #16 0x555cc65afc74 in std::enable_if<is_invocable_r_v<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>, void>::type std::__invoke_r<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #17 0x555cc65afa6c in std::_Function_handler<void (), doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #18 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #19 0x555cca138dab in doris::Thread::supervise_thread(void*) /mnt/disk1/xiaolei/incubator-doris/be/src/util/thread.cpp:498:5 #20 0x555cc645de0a in asan_thread_start(void*) crtstuff.c #21 0x7fef061601c9 in start_thread (/lib64/libpthread.so.0+0x81c9) (BuildId: 823fccea3475e5870a4167dfe47df20e53222db0) #22 0x7fef06b4fe72 in clone (/lib64/libc.so.6+0x39e72) (BuildId: ec3d7025354f1f1985831ff08ef0eb3b50aefbce) 0x5030012ea880 is located 0 bytes inside of 31-byte region [0x5030012ea880,0x5030012ea89f) freed by thread T2249 (REPORT_TASK-182) here: #0 0x555cc649d43d in operator delete(void*) (/mnt/disk1/xiaolei/incubator-doris/output/be/lib/doris_be+0x22cef43d) (BuildId: 76fba423c52f4da5) #1 0x555cc64b6bbc in std::__new_allocator<char>::deallocate(char*, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/new_allocator.h:168:2 #2 0x555cc64b6b5d in std::allocator<char>::deallocate(char*, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/allocator.h:210:25 #3 0x555cc64b6b5d in std::allocator_traits<std::allocator<char>>::deallocate(std::allocator<char>&, char*, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/alloc_traits.h:516:13 #4 0x555cc64b6b5d in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_destroy(unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:289:9 #5 0x555cc64b6a56 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_dispose() /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:283:4 #6 0x555cc64c51ee in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:285:8 #7 0x555cc64c5070 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:1596:8 #8 0x555cc64b1d9c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:802:15 #9 0x555ccb058510 in doris::TBackend::__set_host(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/incubator-doris/gensrc/build/gen_cpp/Types_types.cpp:5584:14 #10 0x555cc9c92239 in doris::BackendOptions::get_local_backend() /mnt/disk1/xiaolei/incubator-doris/be/src/service/backend_options.cpp:75:14 #11 0x555cc6585b9d in doris::report_task_callback(doris::TMasterInfo const&) /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:1004:27 #12 0x555cc64f24b9 in doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/agent_server.cpp:187:112 #13 0x555cc64f2474 in void std::__invoke_impl<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20&>(std::__invoke_other, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #14 0x555cc64f2424 in std::enable_if<is_invocable_r_v<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20&>, void>::type std::__invoke_r<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20&>(doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #15 0x555cc64f22ec in std::_Function_handler<void (), doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_20>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #16 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #17 0x555cc65b01f9 in doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:689:13 #18 0x555cc65afcd4 in void std::__invoke_impl<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(std::__invoke_other, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #19 0x555cc65afc74 in std::enable_if<is_invocable_r_v<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>, void>::type std::__invoke_r<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #20 0x555cc65afa6c in std::_Function_handler<void (), doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #21 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #22 0x555cca138dab in doris::Thread::supervise_thread(void*) /mnt/disk1/xiaolei/incubator-doris/be/src/util/thread.cpp:498:5 #23 0x555cc645de0a in asan_thread_start(void*) crtstuff.c previously allocated by thread T2250 (REPORT_DISK_STA) here: #0 0x555cc649cbdd in operator new(unsigned long) (/mnt/disk1/xiaolei/incubator-doris/output/be/lib/doris_be+0x22ceebdd) (BuildId: 76fba423c52f4da5) #1 0x555cc64b684e in std::__new_allocator<char>::allocate(unsigned long, void const*) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/new_allocator.h:147:27 #2 0x555cc64b67b0 in std::allocator<char>::allocate(unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/allocator.h:198:32 #3 0x555cc64b67b0 in std::allocator_traits<std::allocator<char>>::allocate(std::allocator<char>&, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/alloc_traits.h:482:20 #4 0x555cc64b67b0 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_S_allocate(std::allocator<char>&, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:126:16 #5 0x555cc64b63b1 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_create(unsigned long&, unsigned long) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:155:14 #6 0x555cc64c51de in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:284:24 #7 0x555cc64c5070 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:1596:8 #8 0x555cc64b1d9c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:802:15 #9 0x555ccb058510 in doris::TBackend::__set_host(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk1/xiaolei/incubator-doris/gensrc/build/gen_cpp/Types_types.cpp:5584:14 #10 0x555cc9c92239 in doris::BackendOptions::get_local_backend() /mnt/disk1/xiaolei/incubator-doris/be/src/service/backend_options.cpp:75:14 #11 0x555cc6586e5a in doris::report_disk_callback(doris::StorageEngine&, doris::TMasterInfo const&) /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:1021:27 #12 0x555cc64f2a0a in doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/agent_server.cpp:190:133 #13 0x555cc64f2994 in void std::__invoke_impl<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>(std::__invoke_other, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #14 0x555cc64f2944 in std::enable_if<is_invocable_r_v<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>, void>::type std::__invoke_r<void, doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&>(doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #15 0x555cc64f280c in std::_Function_handler<void (), doris::AgentServer::start_workers(doris::StorageEngine&, doris::ExecEnv*)::$_21>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #16 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #17 0x555cc65b01f9 in doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0::operator()() const /mnt/disk1/xiaolei/incubator-doris/be/src/agent/task_worker_pool.cpp:689:13 #18 0x555cc65afcd4 in void std::__invoke_impl<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(std::__invoke_other, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #19 0x555cc65afc74 in std::enable_if<is_invocable_r_v<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>, void>::type std::__invoke_r<void, doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&>(doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #20 0x555cc65afa6c in std::_Function_handler<void (), doris::ReportWorker::ReportWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::TMasterInfo const&, int, std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #21 0x555cc6612ef2 in std::function<void ()>::operator()() const /mnt/disk1/xiaolei/ldb_17/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #22 0x555cca138dab in doris::Thread::supervise_thread(void*) /mnt/disk1/xiaolei/incubator-doris/be/src/util/thread.cpp:498:5 #23 0x555cc645de0a in asan_thread_start(void*) crtstuff.c ```
yiguolei
pushed a commit
that referenced
this pull request
Jul 19, 2024
…out range of _buf (apache#37936) if we select nested type such as map/array/struct after large string , when string type in mysql_row_buf reserve make buffer size is not large enough , which will lead nested type open_dynamic_mode make _pos pointer out range of mysql_row_buf, then nested type call push_string, and reserve() will make heap_buffer_overflow ``` ==200769==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62d0051c12ba at pc 0x55a77788692d bp 0x7fb52f474a30 sp 0x7fb52f4741f8 READ of size 36541 at 0x62d0051c12ba thread T2309 (Pipe_normal [wo) #0 0x55a77788692c in __asan_memcpy (/mnt/disk1/wangqiannan/amory/doris/output/be/lib/doris_be+0x60c1c92c) (BuildId: 4513940b6b9e22fa) #1 0x55a7a1f622fa in doris::MysqlRowBuffer<false>::reserve(long) /mnt/disk1/wangqiannan/amory/doris/be/src/util/mysql_row_buffer.cpp:140:5 #2 0x55a7a1f638eb in doris::MysqlRowBuffer<false>::push_string(char const*, long) /mnt/disk1/wangqiannan/amory/doris/be/src/util/mysql_row_buffer.cpp:473:5 #3 0x55a7a21f16eb in doris::Status doris::vectorized::DataTypeMapSerDe::_write_column_to_mysql<false>(doris::vectorized::IColumn const&, doris::MysqlRowBuffer<false>&, int, bool, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_map_serde.cpp:410:21 #4 0x55a7a21e4c1e in doris::vectorized::DataTypeMapSerDe::write_column_to_mysql(doris::vectorized::IColumn const&, doris::MysqlRowBuffer<false>&, int, bool, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_map_serde.cpp:478:12 #5 0x55a7a22070e6 in doris::Status doris::vectorized::DataTypeNullableSerDe::_write_column_to_mysql<false>(doris::vectorized::IColumn const&, doris::MysqlRowBuffer<false>&, int, bool, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_nullable_serde.cpp:300:9 #6 0x55a7a21fbc5e in doris::vectorized::DataTypeNullableSerDe::write_column_to_mysql(doris::vectorized::IColumn const&, doris::MysqlRowBuffer<false>&, int, bool, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_nullable_serde.cpp:317:12 #7 0x55a7c2e97e6c in doris::vectorized::VMysqlResultWriter<false>::write(doris::RuntimeState*, doris::vectorized::Block&) /mnt/disk1/wangqiannan/amory/doris/be/src/vec/sink/vmysql_result_writer.cpp:216:17 #8 0x55a7c8031b83 in doris::pipeline::ResultSinkOperatorX::sink(doris::RuntimeState*, doris::vectorized::Block*, bool) /mnt/disk1/wangqiannan/amory/doris/be/src/pipeline/exec/result_sink_operator.cpp:142:5 #9 0x55a7c99a81d6 in doris::pipeline::PipelineTask::execute(bool*)::$_1::operator()() const /mnt/disk1/wangqiannan/amory/doris/be/src/pipeline/pipeline_task.cpp:361:38 #10 0x55a7c99a4b27 in doris::pipeline::PipelineTask::execute(bool*) /mnt/disk1/wangqiannan/amory/doris/be/src/pipeline/pipeline_task.cpp:364:22 #11 0x55a7c9a23a2b in doris::pipeline::TaskScheduler::_do_work(unsigned long) /mnt/disk1/wangqiannan/amory/doris/be/src/pipeline/task_scheduler.cpp:138:9 #12 0x55a7c9a269ca in doris::pipeline::TaskScheduler::start()::$_0::operator()() const /mnt/disk1/wangqiannan/amory/doris/be/src/pipeline/task_scheduler.cpp:64:9 #13 0x55a7c9a2694e in void std::__invoke_impl<void, doris::pipeline::TaskScheduler::start()::$_0&>(std::__invoke_other, doris::pipeline::TaskScheduler::start()::$_0&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #14 0x55a7c9a268ae in std::enable_if<is_invocable_r_v<void, doris::pipeline::TaskScheduler::start()::$_0&>, void>::type std::__invoke_r<void, doris::pipeline::TaskScheduler::start()::$_0&>(doris::pipeline::TaskScheduler::start()::$_0&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #15 0x55a7c9a26635 in std::_Function_handler<void (), doris::pipeline::TaskScheduler::start()::$_0>::_M_invoke(std::_Any_data const&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #16 0x55a777b226da in std::function<void ()>::operator()() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 #17 0x55a77e95ec94 in doris::FunctionRunnable::run() /mnt/disk1/wangqiannan/amory/doris/be/src/util/threadpool.cpp:48:27 #18 0x55a77e941015 in doris::ThreadPool::dispatch_thread() /mnt/disk1/wangqiannan/amory/doris/be/src/util/threadpool.cpp:543:24 #19 0x55a77e97eb23 in void std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(std::__invoke_memfun_deref, void (doris::ThreadPool::*&)(), doris::ThreadPool*&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74:14 #20 0x55a77e97e928 in std::__invoke_result<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 #21 0x55a77e97e860 in void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::__call<void, 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:420:11 #22 0x55a77e97e655 in void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::operator()<void>() /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:503:17 #23 0x55a77e97e54e in void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #24 0x55a77e97e48e in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #25 0x55a77e97dd85 in std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>>::_M_invoke(std::_Any_data const&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #26 0x55a777b226da in std::function<void ()>::operator()() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 #27 0x55a77e8fb841 in doris::Thread::supervise_thread(void*) /mnt/disk1/wangqiannan/amory/doris/be/src/util/thread.cpp:498:5 #28 0x7fc1c3a111c9 in start_thread (/lib64/libpthread.so.0+0x81c9) (BuildId: 823fccea3475e5870a4167dfe47df20e53222db0) #29 0x7fc1c4400e72 in clone (/lib64/libc.so.6+0x39e72) (BuildId: ec3d7025354f1f1985831ff08ef0eb3b50aefbce) 0x62d0051c12ba is located 0 bytes after 36538-byte region [0x62d0051b8400,0x62d0051c12ba) allocated by thread T2309 (Pipe_normal [wo) here: #0 0x55a7778c20bd in operator new[](unsigned long) (/mnt/disk1/wangqiannan/amory/doris/output/be/lib/doris_be+0x60c580bd) (BuildId: 4513940b6b9e22fa) #1 0x55a7a1f621c1 in doris::MysqlRowBuffer<false>::reserve(long) /mnt/disk1/wangqiannan/amory/doris/be/src/util/mysql_row_buffer.cpp:137:21 #2 0x55a7a1f638eb in doris::MysqlRowBuffer<false>::push_string(char const*, long) /mnt/disk1/wangqiannan/amory/doris/be/src/util/mysql_row_buffer.cpp:473:5 #3 0x55a7a1fd0d75 in doris::Status doris::vectorized::DataTypeStringSerDeBase<doris::vectorized::ColumnStr<unsigned int>>::_write_column_to_mysql<false>(doris::vectorized::IColumn const&, doris::MysqlRowBuffer<false>&, int, bool, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_string_serde.h:260:16 #4 0x55a7a1fccc1e in doris::vectorized::DataTypeStringSerDeBase<doris::vectorized::ColumnStr<unsigned int>>::write_column_to_mysql(doris::vectorized::IColumn const&, doris::MysqlRowBuffer<false>&, int, bool, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_string_serde.h:215:16 #5 0x55a7c2e97e6c in doris::vectorized::VMysqlResultWriter<false>::write(doris::RuntimeState*, doris::vectorized::Block&) /mnt/disk1/wangqiannan/amory/doris/be/src/vec/sink/vmysql_result_writer.cpp:216:17 #6 0x55a7c8031b83 in doris::pipeline::ResultSinkOperatorX::sink(doris::RuntimeState*, doris::vectorized::Block*, bool) /mnt/disk1/wangqiannan/amory/doris/be/src/pipeline/exec/result_sink_operator.cpp:142:5 #7 0x55a7c99a81d6 in doris::pipeline::PipelineTask::execute(bool*)::$_1::operator()() const /mnt/disk1/wangqiannan/amory/doris/be/src/pipeline/pipeline_task.cpp:361:38 ``` ## Proposed changes Issue Number: close #xxx <!--Describe your changes.-->
CalvinKirs
pushed a commit
that referenced
this pull request
Aug 15, 2024
…#39201) ## Proposed changes pick apache#39191 Undefined behavior occurs if there is a null value in the list. ``` /root/doris/be/src/vec/common/string_ref.h:271:54: runtime error: null pointer passed as argument 2, which is declared to never be null /var/local/ldb-toolchain/bin/../usr/include/string.h:64:33: note: nonnull attribute specified here #0 0x5616d072245d in doris::StringRef::eq(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:271:41 #1 0x5616d072245d in doris::StringRef::operator==(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:274:60 #2 0x5616d072245d in doris::FixedContainer::find(doris::StringRef const&) const /root/doris/be/src/exprs/hybrid_set.h:76:36 #3 0x5616d072245d in void doris::StringValueSet>::_find_batch(doris::vectorized::IColumn const&, unsigned long, doris::vectorized::PODArray, 16ul, 15ul> const*, doris::vectorized::PODArray, 16ul, 15ul>&) /root/doris/be/src/exprs/hybrid_set.h:688:63 #4 0x5616d0747857 in doris::vectorized::FunctionIn::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/in.h:170:21 #5 0x5616c741fa3a in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function.h:462:26 #6 0x5616cbb5b650 in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp #7 0x5616cbb4e14e in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:244:12 #8 0x5616cbb4e3c2 in doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:250:12 #9 0x5616c741cd68 in doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.h:190:19 #10 0x5616c74cf712 in doris::vectorized::VInPredicate::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vin_predicate.cpp:130:5 #11 0x5616c740d5c0 in doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector>&) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:183:9 #12 0x5616c740ecf5 in doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:215:12 #13 0x5616c7462e24 in doris::vectorized::VCompoundPred::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vcompound_pred.h:127:38 #14 0x5616c74bccec in doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vexpr_context.cpp:54:5 #15 0x5616c74c1dcc in doris::vectorized::VExprContext::execute_conjuncts(std::vector, std::allocator>> const&, std::vector, 16ul, 15ul>, std::allocator, 16ul, 15ul>>> const*, bool, doris::vectorized::Block*, doris::vectorized::PODArray, 16ul, 15ul>, bool) /root/doris/be/src/vec/exprs/vexpr_context.cpp:169:9 #16 0x5616c74c5108 in doris::vectorized::VExprContext::execute_conjuncts_and_filter_block(std::vector, std::allocator>> const&, doris::vectorized::Block*, std::vector>&, int, doris::vectorized::PODArray, 16ul, 15ul>&) /root/doris/be/src/vec/exprs/vexpr_context.cpp:322:5 #17 0x5616ad8a7f1a in doris::segment_v2::SegmentIterator::_execute_common_expr(unsigned short*, unsigned short&, doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2680:5 #18 0x5616ad89e86e in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2582:25 #19 0x5616ad892f5c in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2315:9 #20 0x5616ad892f5c in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2314:19 #21 0x5616ad6dd9cc in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:44:33 #22 0x5616ad269d67 in doris::BetaRowsetReader::next_block(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/beta_rowset_reader.cpp:380:29 #23 0x5616de6de110 in doris::vectorized::VCollectIterator::Level0Iterator::_refresh() /root/doris/be/src/vec/olap/vcollect_iterator.h #24 0x5616de6c967f in doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:514:24 #25 0x5616de6ca8a6 in doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:493:14 #26 0x5616de6d7008 in doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:692:27 #27 0x5616de6bd200 in doris::vectorized::VCollectIterator::build_heap(std::vector, std::allocator>>&) /root/doris/be/src/vec/olap/vcollect_iterator.cpp:186:9 #28 0x5616de651b6c in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /root/doris/be/src/vec/olap/block_reader.cpp:157:5 #29 0x5616de65526f in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /root/doris/be/src/vec/olap/block_reader.cpp:229:19 #30 0x5616e175a0f9 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /root/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:237:32 #31 0x5616c736ad34 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:236:5 #32 0x5616c736f05e in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:176:21 #33 0x5616c736f05e in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:175:31 #34 0x5616c736f05e in void std::_invoke_impl, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(std::_invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #35 0x5616c736f05e in std::enable_if, std::shared_ptr)::$1::operator()() const::'lambda'()&>, void>::type std::_invoke_r, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #36 0x5616c736f05e in std::_Function_handler, std::shared_ptr)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #37 0x5616aeed6a3b in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:543:24 #38 0x5616aeeae4f7 in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5 #39 0x7f7e663e3ac2 in start_thread nptl/pthread_create.c:442:8 #40 0x7f7e6647584f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /root/doris/be/src/vec/common/string_ref.h:271:54 in ``` ## Proposed changes Issue Number: close #xxx <!--Describe your changes.-->
CalvinKirs
pushed a commit
that referenced
this pull request
Aug 15, 2024
…he#39237) (apache#39266) ## Proposed changes apache#39237 which led to accessing uninitialized data. Moved the profile initialization into the init method. ``` /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:100:9: runtime error: member call on null pointer of type 'doris::RuntimeProfile::Counter' #0 0x563f0fb8b6c9 in doris::pipeline::Exchanger<std::shared_ptr<doris::pipeline::BlockWrapper>>::_dequeue_data(doris::pipeline::LocalExchangeSourceLocalState&, std::shared_ptr<doris::pipeline::BlockWrapper>&, bool*, doris::vectorized::Block*, int) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:100:9 #1 0x563f0fb78a45 in doris::pipeline::LocalMergeSortExchanger::finalize(doris::pipeline::LocalExchangeSourceLocalState&) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:325:16 #2 0x563f09610366 in doris::pipeline::LocalExchangeSharedState::sub_running_source_operators(doris::pipeline::LocalExchangeSourceLocalState&) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/dependency.cpp:196:20 #3 0x563f0fb63709 in doris::pipeline::LocalExchangeSourceLocalState::close(doris::RuntimeState*) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchange_source_operator.cpp:59:24 #4 0x563f09728e80 in doris::pipeline::OperatorXBase::close(doris::RuntimeState*) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/exec/operator.cpp:245:28 #5 0x563f0fd3fb58 in doris::pipeline::PipelineTask::close(doris::Status) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/pipeline_task.cpp:459:28 #6 0x563f0fdb5315 in doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:91:27 #7 0x563f0fdb6573 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:125:13 #8 0x563f0fdb9d6a in doris::pipeline::TaskScheduler::start()::$_0::operator()() const /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:64:9 #9 0x563f0fdb9cee in void std::__invoke_impl<void, doris::pipeline::TaskScheduler::start()::$_0&>(std::__invoke_other, doris::pipeline::TaskScheduler::start()::$_0&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #10 0x563f0fdb9c4e in std::enable_if<is_invocable_r_v<void, doris::pipeline::TaskScheduler::start()::$_0&>, void>::type std::__invoke_r<void, doris::pipeline::TaskScheduler::start()::$_0&>(doris::pipeline::TaskScheduler::start()::$_0&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #11 0x563f0fdb99d5 in std::_Function_handler<void (), doris::pipeline::TaskScheduler::start()::$_0>::_M_invoke(std::_Any_data const&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #12 0x563ebdddc8cf in std::function<void ()>::operator()() const /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #13 0x563ec4bd6db4 in doris::FunctionRunnable::run() /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:48:27 #14 0x563ec4bbc1b5 in doris::ThreadPool::dispatch_thread() /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:543:24 #15 0x563ec4bf9a53 in void std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(std::__invoke_memfun_deref, void (doris::ThreadPool::*&)(), doris::ThreadPool*&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:74:14 #16 0x563ec4bf9858 in std::__invoke_result<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96:14 #17 0x563ec4bf9790 in void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::__call<void, 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:506:11 #18 0x563ec4bf9585 in void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::operator()<void>() /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:591:17 #19 0x563ec4bf947e in void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #20 0x563ec4bf93be in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #21 0x563ec4bf8e55 in std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>>::_M_invoke(std::_Any_data const&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #22 0x563ebdddc8cf in std::function<void ()>::operator()() const /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #23 0x563ec4b78d91 in doris::Thread::supervise_thread(void*) /mnt/disk2/yanxuecheng/doris/be/src/util/thread.cpp:498:5 #24 0x563ebdb2fe0a in asan_thread_start(void*) crtstuff.c #25 0x7feeac9e21c9 in start_thread (/lib64/libpthread.so.0+0x81c9) (BuildId: 823fccea3475e5870a4167dfe47df20e53222db0) #26 0x7feead3d1e72 in clone (/lib64/libc.so.6+0x39e72) (BuildId: ec3d7025354f1f1985831ff08ef0eb3b50aefbce) SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:100:9 in *** Query id: ea174401bc134452-bd8a35522726a96a *** *** is nereids: 1 *** *** tablet id: 0 *** *** Aborted at 1723455006 (unix time) try "date -d @1723455006" if you are using GNU date *** *** Current BE git commitID: c47399c *** *** SIGSEGV address not mapped to object (@0x0) received by PID 3055435 (TID 3060341 OR 0x7fe320acf700) from PID 0; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk2/yanxuecheng/doris/be/src/common/signal_handler.h:421 1# 0x00007FEEAD3E6B50 in /lib64/libc.so.6 2# doris::pipeline::Exchanger<std::shared_ptr<doris::pipeline::BlockWrapper> >::_dequeue_data(doris::pipeline::LocalExchangeSourceLocalState&, std::shared_ptr<doris::pipeline::BlockWrapper>&, bool*, doris::vectorized::Block*, int) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:100 3# doris::pipeline::LocalMergeSortExchanger::finalize(doris::pipeline::LocalExchangeSourceLocalState&) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:325 4# doris::pipeline::LocalExchangeSharedState::sub_running_source_operators(doris::pipeline::LocalExchangeSourceLocalState&) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/dependency.cpp:196 5# doris::pipeline::LocalExchangeSourceLocalState::close(doris::RuntimeState*) in /mnt/disk2/yanxuecheng/doris/output/be/lib/doris_be 6# doris::pipeline::OperatorXBase::close(doris::RuntimeState*) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/exec/operator.cpp:245 7# doris::pipeline::PipelineTask::close(doris::Status) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/pipeline_task.cpp:459 8# doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:91 9# doris::pipeline::TaskScheduler::_do_work(unsigned long) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:125 10# doris::pipeline::TaskScheduler::start()::$_0::operator()() const at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:64 11# void std::__invoke_impl<void, doris::pipeline::TaskScheduler::start()::$_0&>(std::__invoke_other, doris::pipeline::TaskScheduler::start()::$_0&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61 12# std::enable_if<is_invocable_r_v<void, doris::pipeline::TaskScheduler::start()::$_0&>, void>::type std::__invoke_r<void, doris::pipeline::TaskScheduler::start()::$_0&>(doris::pipeline::TaskScheduler::start()::$_0&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117 13# std::_Function_handler<void (), doris::pipeline::TaskScheduler::start()::$_0>::_M_invoke(std::_Any_data const&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290 14# std::function<void ()>::operator()() const at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591 15# doris::FunctionRunnable::run() at /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:48 16# doris::ThreadPool::dispatch_thread() at /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:543 17# void std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(std::__invoke_memfun_deref, void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:74 18# std::__invoke_result<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96 19# void std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:506 20# void std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::operator()<, void>() at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:591 21# void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61 22# std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117 23# std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290 24# std::function<void ()>::operator()() const at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591 25# doris::Thread::supervise_thread(void*) at /mnt/disk2/yanxuecheng/doris/be/src/util/thread.cpp:498 26# asan_thread_start(void*) in /mnt/disk2/yanxuecheng/doris/output/be/lib/doris_be 27# start_thread in /lib64/libpthread.so.0 28# __clone in /lib64/libc.so.6 ``` <!--Describe your changes.--> ## Proposed changes Issue Number: close #xxx <!--Describe your changes.-->
CalvinKirs
pushed a commit
that referenced
this pull request
Aug 19, 2024
## Proposed changes Undefined behavior occurs if there is a null value in the list. ``` /root/doris/be/src/vec/common/string_ref.h:271:54: runtime error: null pointer passed as argument 2, which is declared to never be null /var/local/ldb-toolchain/bin/../usr/include/string.h:64:33: note: nonnull attribute specified here #0 0x5616d072245d in doris::StringRef::eq(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:271:41 #1 0x5616d072245d in doris::StringRef::operator==(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:274:60 #2 0x5616d072245d in doris::FixedContainer::find(doris::StringRef const&) const /root/doris/be/src/exprs/hybrid_set.h:76:36 #3 0x5616d072245d in void doris::StringValueSet>::_find_batch(doris::vectorized::IColumn const&, unsigned long, doris::vectorized::PODArray, 16ul, 15ul> const*, doris::vectorized::PODArray, 16ul, 15ul>&) /root/doris/be/src/exprs/hybrid_set.h:688:63 #4 0x5616d0747857 in doris::vectorized::FunctionIn::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/in.h:170:21 #5 0x5616c741fa3a in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function.h:462:26 #6 0x5616cbb5b650 in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp #7 0x5616cbb4e14e in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:244:12 #8 0x5616cbb4e3c2 in doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:250:12 #9 0x5616c741cd68 in doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.h:190:19 #10 0x5616c74cf712 in doris::vectorized::VInPredicate::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vin_predicate.cpp:130:5 #11 0x5616c740d5c0 in doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector>&) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:183:9 #12 0x5616c740ecf5 in doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:215:12 #13 0x5616c7462e24 in doris::vectorized::VCompoundPred::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vcompound_pred.h:127:38 #14 0x5616c74bccec in doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vexpr_context.cpp:54:5 #15 0x5616c74c1dcc in doris::vectorized::VExprContext::execute_conjuncts(std::vector, std::allocator>> const&, std::vector, 16ul, 15ul>, std::allocator, 16ul, 15ul>>> const*, bool, doris::vectorized::Block*, doris::vectorized::PODArray, 16ul, 15ul>, bool) /root/doris/be/src/vec/exprs/vexpr_context.cpp:169:9 #16 0x5616c74c5108 in doris::vectorized::VExprContext::execute_conjuncts_and_filter_block(std::vector, std::allocator>> const&, doris::vectorized::Block*, std::vector>&, int, doris::vectorized::PODArray, 16ul, 15ul>&) /root/doris/be/src/vec/exprs/vexpr_context.cpp:322:5 #17 0x5616ad8a7f1a in doris::segment_v2::SegmentIterator::_execute_common_expr(unsigned short*, unsigned short&, doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2680:5 #18 0x5616ad89e86e in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2582:25 #19 0x5616ad892f5c in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2315:9 #20 0x5616ad892f5c in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2314:19 #21 0x5616ad6dd9cc in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:44:33 #22 0x5616ad269d67 in doris::BetaRowsetReader::next_block(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/beta_rowset_reader.cpp:380:29 #23 0x5616de6de110 in doris::vectorized::VCollectIterator::Level0Iterator::_refresh() /root/doris/be/src/vec/olap/vcollect_iterator.h #24 0x5616de6c967f in doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:514:24 #25 0x5616de6ca8a6 in doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:493:14 #26 0x5616de6d7008 in doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:692:27 #27 0x5616de6bd200 in doris::vectorized::VCollectIterator::build_heap(std::vector, std::allocator>>&) /root/doris/be/src/vec/olap/vcollect_iterator.cpp:186:9 #28 0x5616de651b6c in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /root/doris/be/src/vec/olap/block_reader.cpp:157:5 #29 0x5616de65526f in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /root/doris/be/src/vec/olap/block_reader.cpp:229:19 #30 0x5616e175a0f9 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /root/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:237:32 #31 0x5616c736ad34 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:236:5 #32 0x5616c736f05e in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:176:21 #33 0x5616c736f05e in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:175:31 #34 0x5616c736f05e in void std::_invoke_impl, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(std::_invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #35 0x5616c736f05e in std::enable_if, std::shared_ptr)::$1::operator()() const::'lambda'()&>, void>::type std::_invoke_r, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #36 0x5616c736f05e in std::_Function_handler, std::shared_ptr)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #37 0x5616aeed6a3b in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:543:24 #38 0x5616aeeae4f7 in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5 #39 0x7f7e663e3ac2 in start_thread nptl/pthread_create.c:442:8 #40 0x7f7e6647584f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /root/doris/be/src/vec/common/string_ref.h:271:54 in ```
CalvinKirs
pushed a commit
that referenced
this pull request
Aug 19, 2024
…he#39237) ## Proposed changes which led to accessing uninitialized data. Moved the profile initialization into the init method. ``` /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:100:9: runtime error: member call on null pointer of type 'doris::RuntimeProfile::Counter' #0 0x563f0fb8b6c9 in doris::pipeline::Exchanger<std::shared_ptr<doris::pipeline::BlockWrapper>>::_dequeue_data(doris::pipeline::LocalExchangeSourceLocalState&, std::shared_ptr<doris::pipeline::BlockWrapper>&, bool*, doris::vectorized::Block*, int) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:100:9 #1 0x563f0fb78a45 in doris::pipeline::LocalMergeSortExchanger::finalize(doris::pipeline::LocalExchangeSourceLocalState&) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:325:16 #2 0x563f09610366 in doris::pipeline::LocalExchangeSharedState::sub_running_source_operators(doris::pipeline::LocalExchangeSourceLocalState&) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/dependency.cpp:196:20 #3 0x563f0fb63709 in doris::pipeline::LocalExchangeSourceLocalState::close(doris::RuntimeState*) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchange_source_operator.cpp:59:24 #4 0x563f09728e80 in doris::pipeline::OperatorXBase::close(doris::RuntimeState*) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/exec/operator.cpp:245:28 #5 0x563f0fd3fb58 in doris::pipeline::PipelineTask::close(doris::Status) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/pipeline_task.cpp:459:28 #6 0x563f0fdb5315 in doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:91:27 #7 0x563f0fdb6573 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:125:13 #8 0x563f0fdb9d6a in doris::pipeline::TaskScheduler::start()::$_0::operator()() const /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:64:9 #9 0x563f0fdb9cee in void std::__invoke_impl<void, doris::pipeline::TaskScheduler::start()::$_0&>(std::__invoke_other, doris::pipeline::TaskScheduler::start()::$_0&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #10 0x563f0fdb9c4e in std::enable_if<is_invocable_r_v<void, doris::pipeline::TaskScheduler::start()::$_0&>, void>::type std::__invoke_r<void, doris::pipeline::TaskScheduler::start()::$_0&>(doris::pipeline::TaskScheduler::start()::$_0&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #11 0x563f0fdb99d5 in std::_Function_handler<void (), doris::pipeline::TaskScheduler::start()::$_0>::_M_invoke(std::_Any_data const&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #12 0x563ebdddc8cf in std::function<void ()>::operator()() const /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #13 0x563ec4bd6db4 in doris::FunctionRunnable::run() /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:48:27 #14 0x563ec4bbc1b5 in doris::ThreadPool::dispatch_thread() /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:543:24 #15 0x563ec4bf9a53 in void std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(std::__invoke_memfun_deref, void (doris::ThreadPool::*&)(), doris::ThreadPool*&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:74:14 #16 0x563ec4bf9858 in std::__invoke_result<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96:14 #17 0x563ec4bf9790 in void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::__call<void, 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:506:11 #18 0x563ec4bf9585 in void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::operator()<void>() /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:591:17 #19 0x563ec4bf947e in void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #20 0x563ec4bf93be in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 #21 0x563ec4bf8e55 in std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>>::_M_invoke(std::_Any_data const&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #22 0x563ebdddc8cf in std::function<void ()>::operator()() const /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #23 0x563ec4b78d91 in doris::Thread::supervise_thread(void*) /mnt/disk2/yanxuecheng/doris/be/src/util/thread.cpp:498:5 #24 0x563ebdb2fe0a in asan_thread_start(void*) crtstuff.c #25 0x7feeac9e21c9 in start_thread (/lib64/libpthread.so.0+0x81c9) (BuildId: 823fccea3475e5870a4167dfe47df20e53222db0) #26 0x7feead3d1e72 in clone (/lib64/libc.so.6+0x39e72) (BuildId: ec3d7025354f1f1985831ff08ef0eb3b50aefbce) SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:100:9 in *** Query id: ea174401bc134452-bd8a35522726a96a *** *** is nereids: 1 *** *** tablet id: 0 *** *** Aborted at 1723455006 (unix time) try "date -d @1723455006" if you are using GNU date *** *** Current BE git commitID: c47399c *** *** SIGSEGV address not mapped to object (@0x0) received by PID 3055435 (TID 3060341 OR 0x7fe320acf700) from PID 0; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk2/yanxuecheng/doris/be/src/common/signal_handler.h:421 1# 0x00007FEEAD3E6B50 in /lib64/libc.so.6 2# doris::pipeline::Exchanger<std::shared_ptr<doris::pipeline::BlockWrapper> >::_dequeue_data(doris::pipeline::LocalExchangeSourceLocalState&, std::shared_ptr<doris::pipeline::BlockWrapper>&, bool*, doris::vectorized::Block*, int) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:100 3# doris::pipeline::LocalMergeSortExchanger::finalize(doris::pipeline::LocalExchangeSourceLocalState&) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:325 4# doris::pipeline::LocalExchangeSharedState::sub_running_source_operators(doris::pipeline::LocalExchangeSourceLocalState&) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/dependency.cpp:196 5# doris::pipeline::LocalExchangeSourceLocalState::close(doris::RuntimeState*) in /mnt/disk2/yanxuecheng/doris/output/be/lib/doris_be 6# doris::pipeline::OperatorXBase::close(doris::RuntimeState*) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/exec/operator.cpp:245 7# doris::pipeline::PipelineTask::close(doris::Status) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/pipeline_task.cpp:459 8# doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:91 9# doris::pipeline::TaskScheduler::_do_work(unsigned long) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:125 10# doris::pipeline::TaskScheduler::start()::$_0::operator()() const at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:64 11# void std::__invoke_impl<void, doris::pipeline::TaskScheduler::start()::$_0&>(std::__invoke_other, doris::pipeline::TaskScheduler::start()::$_0&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61 12# std::enable_if<is_invocable_r_v<void, doris::pipeline::TaskScheduler::start()::$_0&>, void>::type std::__invoke_r<void, doris::pipeline::TaskScheduler::start()::$_0&>(doris::pipeline::TaskScheduler::start()::$_0&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117 13# std::_Function_handler<void (), doris::pipeline::TaskScheduler::start()::$_0>::_M_invoke(std::_Any_data const&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290 14# std::function<void ()>::operator()() const at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591 15# doris::FunctionRunnable::run() at /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:48 16# doris::ThreadPool::dispatch_thread() at /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:543 17# void std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(std::__invoke_memfun_deref, void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:74 18# std::__invoke_result<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96 19# void std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:506 20# void std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::operator()<, void>() at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:591 21# void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61 22# std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117 23# std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290 24# std::function<void ()>::operator()() const at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591 25# doris::Thread::supervise_thread(void*) at /mnt/disk2/yanxuecheng/doris/be/src/util/thread.cpp:498 26# asan_thread_start(void*) in /mnt/disk2/yanxuecheng/doris/output/be/lib/doris_be 27# start_thread in /lib64/libpthread.so.0 28# __clone in /lib64/libc.so.6 ``` <!--Describe your changes.-->
CalvinKirs
pushed a commit
that referenced
this pull request
Sep 4, 2024
In following plan pattern, fragment 11 has 3 child fragments aligning with 3 exchange nodes and the union plan node only have 2 plan children, i.e, the aggr node and the exchange node(4). In current union node's parallelism computing, it used the plan children's size to do iteration to find the max parallel but ignore the last fragment's. Actually, it should use fragment's children(exclude unpartitioned ones) instead of plan node's children to do the iteration. 11:VUNION(703) | |----10:VAGGREGATE (update finalize)(697) | | group by: brand_name[#23], skc[#24] | | sortByGroupKey:false | | cardinality=1 | | final projections: '1', brand_name[#25], skc[#26] | | final project output tuple id: 11 | | distribute expr lists: skc[#24] | | | 9:VHASH JOIN(691) | | join op: INNER JOIN(PARTITIONED)[] | | equal join conjunct: (skc[#19] = skc_code[#12]) | | cardinality=1 | | vec output tuple id: 9 | | output tuple id: 9 | | vIntermediate tuple ids: 8 | | hash output slot ids: 18 19 | | final projections: brand_name[#20], skc[#21] | | final project output tuple id: 9 | | distribute expr lists: skc[#19] | | distribute expr lists: skc_code[#12] | | | |----6:VEXCHANGE | | offset: 0 | | distribute expr lists: skc_code[#12] | | | 8:VEXCHANGE | offset: 0 | distribute expr lists: | 4:VEXCHANGE offset: 0 distribute expr lists:
CalvinKirs
pushed a commit
that referenced
this pull request
Sep 13, 2024
…#40643) ## Proposed changes ### Issue ``` *** is nereids: 1 *** tablet id: 4 Abort at 1725864966 (unix time) try "date -d @1725864966" if you are using GNU date *** *** Set a breakpoint in static void __GI_abort() to debug *** PC: @ 0x7f007fb4090a04 *** SIGSEGV (address not mapped to object 0xa0fa868a41d6) received by PID 404737 (TID 274135 OR 0x7ece29df700) from PID 1755584205; stack trace: *** #0 __GI_raise #1 __GI_abort #2 sig_handler #3 _sigaction #4 JVM_handle_linux_signal #5 _sigaction #6 doris::vectorized::ByteArrayDictDecoder::set_dict(std::unique_ptr<unsigned char[], std::default_delete<unsigned char[]>> &&, int, unsigned long) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/format/parquet/byte_array_dict_decoder.cpp:41 #7 doris::vectorized::ColumnChunkReader::_decode_dict_page() at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp:258 #8 doris::vectorized::ColumnChunkReader::next_page() at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp:105 #9 doris::vectorized::ParquetColumnReader::_read_column_data(doris::vectorized::Block*, bool*) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:508 #10 doris::vectorized::ScalarColumnReader::_next_value(doris::vectorized::ICollumn*, unsigned long, unsigned long*, bool*) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:699 #11 doris::vectorized::RowGroupReader::_read_column_data(doris::vectorized::Block*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> &, std::vector<doris::vectorized::ColumnSelectVector>*, unsigned long, unsigned long*, bool*) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:425 #12 doris::vectorized::RowGroupReader::get_next_block(doris::vectorized::Block*, bool*) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:311 #13 doris::vectorized::ParquetReader::get_next(doris::vectorized::Block*, bool*) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/format/parquet/vparquet_reader.cpp:533 #14 doris::vectorized::VFileScanner::_get_next_reader_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/scan/vfile_scanner.cpp:368 #15 doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/scan/vfile_scanner.cpp:411 #16 doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/scan/vscanner.cpp:431 #17 doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/scan/vscanner.cpp:96 #18 doris::vectorized::ScannerScheduler::submit(doris::vectorized::ScannerContext*, std::shared_ptr<doris::vectorized::ScanTask>) at /mnt/disk1/yy/git/enterprise-core/be/src/vec/exec/scan/scanner_context.cpp:96 #19 doris::Thread::supervise_thread(void*) at /mnt/disk1/yy/git/enterprise-core/be/src/util/thread.cpp:499 #20 start_thread #21 clone in /lib64/libc.so.6 ``` ### Solution It is not known why the parquet dictionary page will be null in this case, causing a crash. This PR adds defensive code to prevent the crash.
CalvinKirs
pushed a commit
that referenced
this pull request
Sep 18, 2024
1. Add a new class MemCounter.
2. MemTracker and MemTrackerLimiter no longer have inheritance and
dependencies.
3. MemTrackerLimiter is used to count process memory, the BE web page
`/mem_tracker` also only displays MemTrackerLimiter.
4. MemTracker is used to count the memory of operators and some data
structures. It is not counted in the process memory and is used for
logic control and profile.
In addition, it seems that the crash is caused by memory abnormality,
not sure whether this PR can fix the problem, but this will help locate
the problem and more elegant.
```
==6641==ERROR: AddressSanitizer: heap-use-after-free on address 0x603000980e70 at pc 0x559be21880be bp 0x7fd7cfc75070 sp 0x7fd7cfc75068
READ of size 8 at 0x603000980e70 thread T1453 (memory_maintena)
#0 0x559be21880bd in std::__atomic_base<long>::load(std::memory_order) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x559be21880bd in doris::MemTracker::MemCounter::current_value() const /home/zcp/repo_center/doris_master/doris/be/src/runtime/memory/mem_tracker.h:139:63
#2 0x559be21880bd in doris::MemTracker::consumption() const /home/zcp/repo_center/doris_master/doris/be/src/runtime/memory/mem_tracker.h:165:56
#3 0x559be3985133 in doris::MemTrackerLimiter::refresh_global_counter() /home/zcp/repo_center/doris_master/doris/be/src/runtime/memory/mem_tracker_limiter.cpp:245:59
#4 0x559bdfcd52d5 in doris::Daemon::memory_maintenance_thread() /home/zcp/repo_center/doris_master/doris/be/src/common/daemon.cpp:239:13
#5 0x559be40e6c17 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
#6 0x7fdf563e3ac2 in start_thread nptl/pthread_create.c:442:8
#7 0x7fdf5647584f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
0x603000980e74 is located 0 bytes after 20-byte region [0x603000980e60,0x603000980e74)
04:19:48 freed by thread T1491 (Scan_normal [wo) here:
04:19:48 #0 0x559bdfb0dd9d in operator delete(void*) (/mnt/hdd01/ci/master-deploy/be/lib/doris_be+0x33546d9d) (BuildId: f170e92ad3c55512)
04:19:48 #1 0x559bdfb1d42b in __gnu_cxx::new_allocator<char>::deallocate(char*, unsigned long) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:139:2
04:19:48 #2 0x559bdfb1d42b in std::allocator<char>::deallocate(char*, unsigned long) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:187:27
04:19:48 #3 0x559bdfb1d42b in std::allocator_traits<std::allocator<char>>::deallocate(std::allocator<char>&, char*, unsigned long) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:492:13
04:19:48 #4 0x559bdfb1d42b in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_destroy(unsigned long) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:237:9
04:19:48 #5 0x559bdfb1d42b in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_dispose() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:232:4
04:19:48 #6 0x559bdfb1d42b in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::~basic_string() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:658:9
04:19:48 #7 0x559bdfb22501 in void std::destroy_at<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
04:19:48 #8 0x559bdfb22501 in void std::_Destroy<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
04:19:48 #9 0x559bdfb22501 in void std::_Destroy_aux<false>::__destroy<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
04:19:48 #10 0x559bdfb22501 in void std::_Destroy<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
04:19:48 #11 0x559bdfb22501 in void std::_Destroy<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
04:19:48 #12 0x559bdfb22501 in std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::~vector() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2
04:19:48 #13 0x559be5515262 in doris::TCondition::~TCondition() /home/zcp/repo_center/doris_master/doris/gensrc/build/gen_cpp/PaloInternalService_types.cpp:9867:1
04:19:48 #14 0x559be5515262 in doris::TCondition::~TCondition() /home/zcp/repo_center/doris_master/doris/gensrc/build/gen_cpp/PaloInternalService_types.cpp:9866:36
04:19:48 #15 0x559be0047ee6 in doris::Status doris::DeleteHandler::_parse_column_pred<doris::DeleteSubPredicatePB>(std::shared_ptr<doris::TabletSchema>, std::shared_ptr<doris::TabletSchema>, google::protobuf::RepeatedPtrField<doris::DeleteSubPredicatePB> const&, doris::DeleteConditions*) /home/zcp/repo_center/doris_master/doris/be/src/olap/delete_handler.cpp:372:5
04:19:48 #16 0x559be003ba59 in doris::DeleteHandler::init(std::shared_ptr<doris::TabletSchema>, std::vector<std::shared_ptr<doris::RowsetMeta>, std::allocator<std::shared_ptr<doris::RowsetMeta>>> const&, long, bool) /home/zcp/repo_center/doris_master/doris/be/src/olap/delete_handler.cpp:404:13
04:19:48 #17 0x559be30ed6e3 in doris::TabletReader::_init_delete_condition(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:636:28
04:19:48 #18 0x559be30e23d1 in doris::TabletReader::_init_params(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:290:18
04:19:48 #19 0x559be30e1592 in doris::TabletReader::init(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:124:18
04:19:48 #20 0x559c129840c8 in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_master/doris/be/src/vec/olap/block_reader.cpp:210:5
04:19:48 #21 0x559c15a4ea09 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:232:32
04:19:48 #22 0x559bfb4acd94 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:236:5
04:19:48 #23 0x559bfb4b10be in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:176:21
04:19:48 #24 0x559bfb4b10be in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:175:31
04:19:48 #25 0x559bfb4b10be in void std::__invoke_impl<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
04:19:48 #26 0x559bfb4b10be in std::enable_if<is_invocable_r_v<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&>, void>::type std::__invoke_r<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
04:19:48 #27 0x559bfb4b10be in std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
04:19:48 #28 0x559be410ed1b in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:543:24
04:19:48 #29 0x559be40e6c17 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
04:19:48 #30 0x7fdf563e3ac2 in start_thread nptl/pthread_create.c:442:8
04:19:48
04:19:48 previously allocated by thread T1491 (Scan_normal [wo) here:
04:19:48 #0 0x559bdfb0d53d in operator new(unsigned long) (/mnt/hdd01/ci/master-deploy/be/lib/doris_be+0x3354653d) (BuildId: f170e92ad3c55512)
04:19:48 #1 0x559bdfb33f74 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_construct<char*>(char*, char*, std::forward_iterator_tag) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:219:14
04:19:48 #2 0x559bdfb32705 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_construct_aux<char*>(char*, char*, std::__false_type) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:247:11
04:19:48 #3 0x559bdfb32705 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_construct<char*>(char*, char*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:266:4
04:19:48 #4 0x559bdfb32705 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:451:9
04:19:48 #5 0x559bdfc326e6 in decltype(::new((void*)(0)) std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>(std::declval<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&>())) std::construct_at<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:97:39
04:19:48 #6 0x559bdfc326e6 in void std::allocator_traits<std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::construct<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&>(std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:514:4
04:19:48 #7 0x559bdfc326e6 in void std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::_M_realloc_insert<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&>(__gnu_cxx::__normal_iterator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/vector.tcc:449:4
04:19:48 #8 0x559be0032206 in doris::DeleteHandler::parse_condition(doris::DeleteSubPredicatePB const&, doris::TCondition*) /home/zcp/repo_center/doris_master/doris/be/src/olap/delete_handler.cpp:300:33
04:19:48 #9 0x559be0047aa3 in doris::Status doris::DeleteHandler::_parse_column_pred<doris::DeleteSubPredicatePB>(std::shared_ptr<doris::TabletSchema>, std::shared_ptr<doris::TabletSchema>, google::protobuf::RepeatedPtrField<doris::DeleteSubPredicatePB> const&, doris::DeleteConditions*) /home/zcp/repo_center/doris_master/doris/be/src/olap/delete_handler.cpp:355:9
04:19:48 #10 0x559be003ba59 in doris::DeleteHandler::init(std::shared_ptr<doris::TabletSchema>, std::vector<std::shared_ptr<doris::RowsetMeta>, std::allocator<std::shared_ptr<doris::RowsetMeta>>> const&, long, bool) /home/zcp/repo_center/doris_master/doris/be/src/olap/delete_handler.cpp:404:13
04:19:48 #11 0x559be30ed6e3 in doris::TabletReader::_init_delete_condition(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:636:28
04:19:48 #12 0x559be30e23d1 in doris::TabletReader::_init_params(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:290:18
04:19:48 #13 0x559be30e1592 in doris::TabletReader::init(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:124:18
04:19:48 #14 0x559c129840c8 in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_master/doris/be/src/vec/olap/block_reader.cpp:210:5
04:19:48 #15 0x559c15a4ea09 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:232:32
04:19:48 #16 0x559bfb4acd94 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:236:5
04:19:48 #17 0x559bfb4b10be in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:176:21
04:19:48 #18 0x559bfb4b10be in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:175:31
04:19:48 #19 0x559bfb4b10be in void std::__invoke_impl<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
04:19:48 #20 0x559bfb4b10be in std::enable_if<is_invocable_r_v<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&>, void>::type std::__invoke_r<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
04:19:48 #21 0x559bfb4b10be in std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
04:19:48 #22 0x559be410ed1b in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:543:24
04:19:48 #23 0x559be40e6c17 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
04:19:48 #24 0x7fdf563e3ac2 in start_thread nptl/pthread_create.c:442:8
```
CalvinKirs
pushed a commit
that referenced
this pull request
Oct 14, 2024
…#41743) ==40376==ERROR: AddressSanitizer: heap-use-after-free on address 0x616002419ed0 at pc 0x56132d2dc34a bp 0x7fe20150ba70 sp 0x7fe20150ba68 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27557_27557&logView=flowAware) WRITE of size 4 at 0x616002419ed0 thread T1474 (Pipe_normal [wo) [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27558_27558&logView=flowAware) #0 0x56132d2dc349 in doris::pipeline::PipelineTask::update_queue_level(int) /root/doris/be/src/pipeline/pipeline_task.h:174:67 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27559_27559&logView=flowAware) #1 0x56132d2dc349 in doris::pipeline::PriorityTaskQueue::_try_take_unprotected(bool) /root/doris/be/src/pipeline/task_queue.cpp:79:15 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27560_27560&logView=flowAware) #2 0x56132d2dc615 in doris::pipeline::PriorityTaskQueue::try_take(bool) /root/doris/be/src/pipeline/task_queue.cpp:97:12 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27561_27561&logView=flowAware) #3 0x56132d2de69f in doris::pipeline::MultiCoreTaskQueue::take(int) /root/doris/be/src/pipeline/task_queue.cpp:164:50 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27562_27562&logView=flowAware) #4 0x56132d2ee0e3 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /root/doris/be/src/pipeline/task_scheduler.cpp:102:35 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27563_27563&logView=flowAware) #5 0x5612fc23029d in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:543:24 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27564_27564&logView=flowAware) #6 0x5612fc20880e in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27565_27565&logView=flowAware) #7 0x5612fc20880e in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27566_27566&logView=flowAware) #8 0x7fe9405ca608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27567_27567&logView=flowAware) #9 0x7fe940877132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27568_27568&logView=flowAware) [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27569_27569&logView=flowAware) 0x616002419ed0 is located 80 bytes inside of 632-byte region [0x616002419e80,0x61600241a0f8) [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27570_27570&logView=flowAware) freed by thread T1463 (Pipe_normal [wo) here: [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27571_27571&logView=flowAware) #0 0x5612f7ac280d in operator delete(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x2ed1180d) (BuildId: 04f75ee9f41a400a) [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27572_27572&logView=flowAware) #1 0x56132d2519d4 in std::default_delete<doris::pipeline::PipelineTask>::operator()(doris::pipeline::PipelineTask*) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27573_27573&logView=flowAware) #2 0x56132d2519d4 in std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >::~unique_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27574_27574&logView=flowAware) #3 0x56132d2519d4 in void std::destroy_at<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27575_27575&logView=flowAware) #4 0x56132d2519d4 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27576_27576&logView=flowAware) #5 0x56132d2519d4 in void std::_Destroy_aux<false>::__destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*>(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27577_27577&logView=flowAware) #6 0x56132d2519d4 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*>(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27578_27578&logView=flowAware) #7 0x56132d2519d4 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27579_27579&logView=flowAware) #8 0x56132d2519d4 in std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >::~vector() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27580_27580&logView=flowAware) #9 0x56132d1d717c in void std::destroy_at<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27581_27581&logView=flowAware) #10 0x56132d1d717c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27582_27582&logView=flowAware) #11 0x56132d1d717c in void std::_Destroy_aux<false>::__destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27583_27583&logView=flowAware) #12 0x56132d1d717c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27584_27584&logView=flowAware) #13 0x56132d1d717c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27585_27585&logView=flowAware) #14 0x56132d1d717c in std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > > >::_M_erase_at_end(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1796:6 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27586_27586&logView=flowAware) #15 0x56132d1d717c in std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > > >::clear() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1499:9 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27587_27587&logView=flowAware) #16 0x56132d1d717c in doris::pipeline::PipelineFragmentContext::~PipelineFragmentContext() /root/doris/be/src/pipeline/pipeline_fragment_context.cpp:143:12 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27588_27588&logView=flowAware) #17 0x5612f7aeabdc in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27589_27589&logView=flowAware) #18 0x56132d2ed705 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:11 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27590_27590&logView=flowAware) #19 0x56132d2ed705 in std::__shared_ptr<doris::TaskExecutionContext, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27591_27591&logView=flowAware) #20 0x56132d2ed705 in doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) /root/doris/be/src/pipeline/task_scheduler.cpp:98:1 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27592_27592&logView=flowAware) #21 0x56132d2efce9 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /root/doris/be/src/pipeline/task_scheduler.cpp:181:17 [02:02:28 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_P0Regression/542356?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&expandBuildChangesSection=true&expandBuildTestsSection=true&showLog=542356_27593_27593&logView=flowAware) #22 0x5612fc23029d in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:543:24
CalvinKirs
pushed a commit
that referenced
this pull request
Oct 22, 2024
…dir when index file writer open index (apache#42207) ## Proposed changes Fix UAF as below ``` ==16442==ERROR: AddressSanitizer: heap-use-after-free on address 0x60f0008718a0 at pc 0x5586663cea7a bp 0x7f02673d0530 sp 0x7f02673d0528 READ of size 8 at 0x60f0008718a0 thread T490 (CumuCompactionT) #0 0x5586663cea79 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::finish() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:549:63 #1 0x5586663360bc in doris::segment_v2::ScalarColumnWriter::write_inverted_index() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:640:41 #2 0x5586662ce160 in doris::segment_v2::SegmentWriter::_write_inverted_index() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:1435:9 #3 0x5586662ccb13 in doris::segment_v2::SegmentWriter::finalize_columns_index(unsigned long*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:1317:5 #4 0x55866653c074 in doris::VerticalBetaRowsetWriter::_flush_columns(doris::segment_v2::SegmentWriter*, bool) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/vertical_beta_rowset_writer.cpp:127:5 #5 0x558666534e3f in doris::VerticalBetaRowsetWriter::flush_columns(bool) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/vertical_beta_rowset_writer.cpp:153:5 #6 0x558665a8d8b2 in doris::Merger::vertical_compact_one_group(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, bool, std::vector> const&, doris::vectorized::RowSourcesBuffer*, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, doris::Merger::Statistics*, std::vector>, long, doris::CompactionSampleInfo*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:333:5 #7 0x558665a93190 in doris::Merger::vertical_merge_rowsets(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, long, doris::Merger::Statistics*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:471:21 #8 0x558665a00fd7 in doris::Compaction::merge_input_rowsets() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:192:19 #9 0x558665a329a3 in doris::CloudCompactionMixin::execute_compact_impl(long) /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1158:5 #10 0x558665a33381 in doris::CloudCompactionMixin::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1173:5 #11 0x55869a893d89 in doris::CloudCumulativeCompaction::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_cumulative_compaction.cpp:191:38 #12 0x55869a861fac in doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_storage_engine.cpp:693:31 #13 0x55869a861fac in void std::__invoke_impl const&)::$_1&>(std::__invoke_other, doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #14 0x55869a861fac in std::enable_if const&)::$_1&>, void>::type std::__invoke_r const&)::$_1&>(doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #15 0x55869a861fac in std::_Function_handler const&)::$_1>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #16 0x5586679033fb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:543:24 #17 0x5586678db057 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #18 0x7f0639d7bac2 in start_thread nptl/pthread_create.c:442:8 #19 0x7f0639e0d84f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 0x60f0008718a0 is located 0 bytes inside of 168-byte region [0x60f0008718a0,0x60f000871948) freed by thread T490 (CumuCompactionT) here: #0 0x558663259d9d in operator delete(void*) (/mnt/hdd01/ci/master-deploy/cluster0/be/lib/doris_be+0x337a1d9d) (BuildId: fa8094411569cb8d) #1 0x558665a655eb in std::default_delete::operator()(lucene::store::Directory*) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2 #2 0x558665a655eb in std::unique_ptr>::~unique_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4 #3 0x558665f83dfe in std::pair, std::allocator>> const, std::unique_ptr>>::~pair() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_iterator.h:2379:12 #4 0x558665f83dfe in void std::destroy_at, std::allocator>> const, std::unique_ptr>>>(std::pair, std::allocator>> const, std::unique_ptr>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15 #5 0x558665f83dfe in void std::allocator_traits, std::allocator>> const, std::unique_ptr>>>>>::destroy, std::allocator>> const, std::unique_ptr>>>(std::allocator, std::allocator>> const, std::unique_ptr>>>>&, std::pair, std::allocator>> const, std::unique_ptr>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:533:4 #6 0x558665f83dfe in std::_Rb_tree, std::allocator>>, std::pair, std::allocator>> const, std::unique_ptr>>, std::_Select1st, std::allocator>> const, std::unique_ptr>>>, std::less, std::allocator>>>, std::allocator, std::allocator>> const, std::unique_ptr>>>>::_M_destroy_node(std::_Rb_tree_node, std::allocator>> const, std::unique_ptr>>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:623:2 #7 0x558665f83dfe in std::_Rb_tree, std::allocator>>, std::pair, std::allocator>> const, std::unique_ptr>>, std::_Select1st, std::allocator>> const, std::unique_ptr>>>, std::less, std::allocator>>>, std::allocator, std::allocator>> const, std::unique_ptr>>>>::_M_drop_node(std::_Rb_tree_node, std::allocator>> const, std::unique_ptr>>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:631:2 #8 0x558665f9e9b8 in std::pair, std::allocator>> const, std::unique_ptr>>>, bool> std::_Rb_tree, std::allocator>>, std::pair, std::allocator>> const, std::unique_ptr>>, std::_Select1st, std::allocator>> const, std::unique_ptr>>>, std::less, std::allocator>>>, std::allocator, std::allocator>> const, std::unique_ptr>>>>::_M_emplace_unique, std::allocator>>, std::unique_ptr>>(std::pair, std::allocator>>&&, std::unique_ptr>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:2391:6 #9 0x558665f86e29 in std::pair, std::allocator>> const, std::unique_ptr>>>, bool> std::map, std::allocator>>, std::unique_ptr>, std::less, std::allocator>>>, std::allocator, std::allocator>> const, std::unique_ptr>>>>::emplace, std::allocator>>, std::unique_ptr>>(std::pair, std::allocator>>&&, std::unique_ptr>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_map.h:577:16 #10 0x558665f86e29 in doris::segment_v2::InvertedIndexFileWriter::open(doris::TabletIndex const*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_file_writer.cpp:60:19 #11 0x5586663d0423 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::open_index_directory() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:160:16 #12 0x5586663d00f7 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::init_bkd_index() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:146:16 #13 0x5586663cb2d9 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::init() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:107:24 #14 0x5586663b1201 in doris::segment_v2::InvertedIndexColumnWriter::create(doris::Field const*, std::unique_ptr>*, doris::segment_v2::InvertedIndexFileWriter*, doris::TabletIndex const*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:697:27 #15 0x55866632cf8d in doris::segment_v2::ScalarColumnWriter::init() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:483:13 #16 0x5586662a5e52 in doris::segment_v2::SegmentWriter::_create_column_writer(unsigned int, doris::TabletColumn const&, std::shared_ptr const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:267:5 #17 0x5586662a71b4 in doris::segment_v2::SegmentWriter::_create_writers(std::shared_ptr const&, std::vector> const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:316:9 #18 0x5586662a2be3 in doris::segment_v2::SegmentWriter::init(std::vector> const&, bool) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:285:5 #19 0x5586665333b2 in doris::VerticalBetaRowsetWriter::add_columns(doris::vectorized::Block const*, std::vector> const&, bool, unsigned int) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/vertical_beta_rowset_writer.cpp:96:17 #20 0x558665a8cb9b in doris::Merger::vertical_compact_one_group(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, bool, std::vector> const&, doris::vectorized::RowSourcesBuffer*, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, doris::Merger::Statistics*, std::vector>, long, doris::CompactionSampleInfo*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:309:9 #21 0x558665a93190 in doris::Merger::vertical_merge_rowsets(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, long, doris::Merger::Statistics*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:471:21 #22 0x558665a00fd7 in doris::Compaction::merge_input_rowsets() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:192:19 #23 0x558665a329a3 in doris::CloudCompactionMixin::execute_compact_impl(long) /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1158:5 #24 0x558665a33381 in doris::CloudCompactionMixin::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1173:5 #25 0x55869a893d89 in doris::CloudCumulativeCompaction::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_cumulative_compaction.cpp:191:38 #26 0x55869a861fac in doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_storage_engine.cpp:693:31 #27 0x55869a861fac in void std::__invoke_impl const&)::$_1&>(std::__invoke_other, doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #28 0x55869a861fac in std::enable_if const&)::$_1&>, void>::type std::__invoke_r const&)::$_1&>(doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #29 0x55869a861fac in std::_Function_handler const&)::$_1>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #30 0x5586679033fb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:543:24 #31 0x5586678db057 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #32 0x7f0639d7bac2 in start_thread nptl/pthread_create.c:442:8 previously allocated by thread T490 (CumuCompactionT) here: #0 0x55866325953d in operator new(unsigned long) (/mnt/hdd01/ci/master-deploy/cluster0/be/lib/doris_be+0x337a153d) (BuildId: fa8094411569cb8d) #1 0x558665fb7d7b in doris::segment_v2::DorisFSDirectoryFactory::getDirectory(std::shared_ptr const&, char const*, bool, lucene::store::LockFactory*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_fs_directory.cpp:797:15 #2 0x558665f86b5c in doris::segment_v2::InvertedIndexFileWriter::open(doris::TabletIndex const*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_file_writer.cpp:58:17 #3 0x5586663d0423 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::open_index_directory() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:160:16 #4 0x5586663d00f7 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::init_bkd_index() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:146:16 #5 0x5586663cb2d9 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::init() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:107:24 #6 0x5586663b1201 in doris::segment_v2::InvertedIndexColumnWriter::create(doris::Field const*, std::unique_ptr>*, doris::segment_v2::InvertedIndexFileWriter*, doris::TabletIndex const*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:697:27 #7 0x55866632cf8d in doris::segment_v2::ScalarColumnWriter::init() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:483:13 #8 0x5586662a5e52 in doris::segment_v2::SegmentWriter::_create_column_writer(unsigned int, doris::TabletColumn const&, std::shared_ptr const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:267:5 #9 0x5586662a71b4 in doris::segment_v2::SegmentWriter::_create_writers(std::shared_ptr const&, std::vector> const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:316:9 #10 0x5586662a2be3 in doris::segment_v2::SegmentWriter::init(std::vector> const&, bool) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:285:5 #11 0x5586665333b2 in doris::VerticalBetaRowsetWriter::add_columns(doris::vectorized::Block const*, std::vector> const&, bool, unsigned int) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/vertical_beta_rowset_writer.cpp:96:17 #12 0x558665a8cb9b in doris::Merger::vertical_compact_one_group(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, bool, std::vector> const&, doris::vectorized::RowSourcesBuffer*, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, doris::Merger::Statistics*, std::vector>, long, doris::CompactionSampleInfo*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:309:9 #13 0x558665a93190 in doris::Merger::vertical_merge_rowsets(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, long, doris::Merger::Statistics*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:471:21 #14 0x558665a00fd7 in doris::Compaction::merge_input_rowsets() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:192:19 #15 0x558665a329a3 in doris::CloudCompactionMixin::execute_compact_impl(long) /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1158:5 #16 0x558665a33381 in doris::CloudCompactionMixin::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1173:5 #17 0x55869a893d89 in doris::CloudCumulativeCompaction::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_cumulative_compaction.cpp:191:38 #18 0x55869a861fac in doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_storage_engine.cpp:693:31 #19 0x55869a861fac in void std::__invoke_impl const&)::$_1&>(std::__invoke_other, doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #20 0x55869a861fac in std::enable_if const&)::$_1&>, void>::type std::__invoke_r const&)::$_1&>(doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #21 0x55869a861fac in std::_Function_handler const&)::$_1>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #22 0x5586679033fb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:543:24 #23 0x5586678db057 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #24 0x7f0639d7bac2 in start_thread nptl/pthread_create.c:442:8 ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Issue Number: close #xxx
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...